Consumers have the need for producing new images from their original digital images. A collage is a popular form of such new images.
Collage (From the French: coller, to stick) is regarded as a work of visual arts made from an assemblage of different forms or parts, thus creating a new whole. The parts are related to each other either for the same event or theme. This term was coined by both Georges Braque and Pablo Picasso in the beginning of the 20th century when collage became a distinctive part of modern art. Use of this technique made its dramatic appearance among oil paintings in the early 20th century as an art form of groundbreaking novelty.
An artistic collage work can include newspaper clippings, ribbons, bits of colored or hand-made papers, portions of other artwork, photographs, and such, glued to a piece of paper or canvas.
Techniques of collage were first used at the time of the invention of paper in China around 200 BC. The use of collage, however, remained very limited until the 10th century in Japan, when calligraphers began to apply glued paper, using texts on surfaces, when writing their poems.
Since the beginning of the 19th century, collage methods also were used among hobbyists for memorabilia (i.e. applied to photo albums) and books (i.e. Hans Christian Andersen, Carl Spitzweg). For example, a consumer can create a collage of a special event, e.g., a NBA all-star game, by combining personal photos with professional photos and/or background.
Digital collage is the technique of using computer tools in collage creation to encourage chance associations of disparate visual elements and the subsequent transformation of the visual results through the use of electronic media.
A collage made from photographs, or parts of photographs, is also called photomontage. Photomontage is the process (and result) of making a composite photograph by cutting and joining a number of other photographs. The composite picture was sometimes photographed so that the final image is converted back into a seamless photographic print. The same method is accomplished today using image-editing software. The technique is referred to by professionals as “compositing”.
Creating a photomontage has, for the most part, become easier with the advent of computer software such as Adobe Photoshop. These programs make the changes digitally, allowing for faster workflow and more precise results. They also mitigate mistakes by allowing the artist to “undo” errors.
For casual users such as consumers, it is desirable to automate the process of creating a collage because they are usually not skilled at digital image editing and it is labor intensive to cut out the portions of interest from the images. A number of existing image editing software products provides automatic collages. A simple solution is simply tiling individual images into a collage; the resulted collage is usually not interesting (see “tiling” in FIG. 4). Google Picasa piles photos on top of each other with no regard to the scene content. The results are usually not satisfactory (see “piling” in FIG. 5) because the most interesting parts of images can be blocked by other images. The source of the problem is the lack of scene analysis to understand where the main subject and background are in the image.
There are other products in the market: AKVIS Chameleon, Wondershare Photo Collage Studio, fCoder Group PhotoMix, Three Dot Lab EasyCollage, VickMan Photo Collage Screensaver, iFoxSoft Photo Collage, and iPhoto. None of the tools exhibit automated main subject detection to cutout or place main subject into products. In comparison, iFoxSoft QuickSnap and ArcSoft Cut-It-Out provide main subject extraction by having the user manually mark the desired subject and background areas of the image.
To make smart collages, Wang et al. (Jindong Wang, Jian Sun, Long Quan, Xiaouou Tang and Heung-Yeung Shum. Picture Collage. CVPR 2006, Vol. 1, pp. 347-354.) use attention estimation to decide a region of interest (ROI) and then use an optimization technique known as graph cuts to perform layout so as to avoid covering the ROIs. However, Wang et al. assume that there is a single primary ROI in each image and the attention estimation always finds the ROI, neither of which is guaranteed. In addition, Wang et al. do not produce cutouts from the source images; the source images are laid out on top of each other, albeit with the ROIs visible (see “staggering” in FIG. 6).
Rother et al. (Carsten Rother, Lucas Bordeaux, Youssef Hamadi, Andrew Blake, AutoCollage, ACM Transactions on Graphics (SIGGRAPH), August 2006, Pages: 847-852.) consider sky and grass regions such that the photos with prominent sky regions appear at the top and those with prominent grass regions appear at the bottom of the collage. Also note this approach generally does not involve cut-out, or overlay; it blends all the images together (see “blending” in FIG. 7).
Neither of these techniques involves special treatment of the most privileged regions in the images. In most cases, the most privileged part of an image is the face or faces while sometimes animals, flowers or any other peculiar object can be the main subject of an image and thus needs to be preserved during the collage process. Moreover, it is often desirable to retain some margin space around such main subjects such that they are not cut out too tight. In the case of faces, we refer to such margin as “head room”.
There is a need to provide an automatic, intelligent, and reliable process for identifying the main subject for cutout or optimum placement on a collage or in a template hole, so that (1) the main subject of the image is not cropped in part or in its entirety, (2) both smooth and textured background can be identified and excluded in part or in its entirety if necessary, and (3) common picture composition rules such as sufficient headroom can be enforced.